Papers with variational autoencoders
Controlled Text Generation Using Dictionary Prior in Variational Autoencoders (2022.findings-acl)
Copied to clipboard
| Challenge: | Variational autoencoders (VAEs) have been widely applied in text generation tasks, but they suffer from insufficient representation capacity and poor controllability. |
| Approach: | They propose a data-driven prior that has expressivity and controllability. |
| Outcome: | The proposed prior enjoys expressivity and controllability and can be used in language modeling and controlled text generation. |
LangVAE and LangSpace: Building and Probing for Language Model VAEs (2025.emnlp-demos)
Copied to clipboard
| Challenge: | a novel framework for modular construction of variational autoencoders (VAEs) on pre-trained large language models (LLMs) is presented. |
| Approach: | They propose a modular framework for variational autoencoders on top of pre-trained large language models. |
| Outcome: | The proposed framework can encode pre-trained language models into more compact and semantically disentangled representations. |
Disentangling Representations of Text by Masking Transformers (2021.emnlp-main)
Copied to clipboard
| Challenge: | Large pretrained models such as BERT encode a range of features into monolithic vectors, providing strong predictive accuracy across downstream tasks. |
| Approach: | They explore whether it is possible to learn disentangled representations by identifying existing subnetworks within pretrained models that encode distinct, complementary aspects. |
| Outcome: | The proposed method disentangles sentiment from genre in movie reviews, toxicity from dialect in Tweets, and syntax from semantics. |
Generative Text Modeling through Short Run Inference (2021.eacl-main)
Copied to clipboard
| Challenge: | Latent variable models for text capture global semantic and syntactic features when trained correctly. |
| Approach: | They propose a short run dynamics for inference that initializes from the prior distribution of the latent variable and runs a small number of Langevin dynamics steps guided by its posterior distribution. |
| Outcome: | The proposed model is able to generate coherent sentences with smooth transition and shows no sign of posterior collapse. |
Unsupervised Discrete Sentence Representation Learning for Interpretable Neural Dialog Generation (P18-1)
Copied to clipboard
| Challenge: | Existing encoder-decoder dialog models cannot output interpretable actions as in traditional systems. |
| Approach: | They propose an unsupervised discrete sentence representation learning method that integrates with existing encoder-decoder dialog models for interpretable response generation. |
| Outcome: | The proposed model can be integrated with existing encoder-decoder dialog models and discover interpretable semantics via either auto encoding or context predicting. |
Training Data Augmentation for Detecting Adverse Drug Reactions in User-Generated Content (D19-1)
Copied to clipboard
Sepideh Mesbah, Jie Yang, Robert-Jan Sips, Manuel Valle Torre, Christoph Lofi, Alessandro Bozzon, Geert-Jan Houben
| Challenge: | Existing dictionary-based, semi-supervised learning approaches are limited by the coverage and maintainability of laymen health vocabularies. |
| Approach: | They propose a data augmentation approach that leverages variational autoencoders to learn high-quality data distributions from a large unlabeled dataset and generate a small set of labeled training sets. |
| Outcome: | The proposed approach matches the performance of fully-supervised approaches while requiring only 25% of training data. |
vONTSS: vMF based semi-supervised neural topic modeling with optimal transport (2023.findings-acl)
Copied to clipboard
| Challenge: | Recent Neural Topic Models (NTMs) have limited applications in the real world due to the challenge of incorporating human knowledge. |
| Approach: | They propose a semi-supervised neural topic modeling method, vONTSS, which uses von Mises-Fisher variational autoencoders and optimal transport. |
| Outcome: | The proposed method outperforms existing semi-supervised topic modeling methods on multiple aspects. |
Perceptual Models of Machine-Edited Text (2021.findings-acl)
Copied to clipboard
| Challenge: | a dataset of human judgments of machine-edited text is presented . we compare six different methods to create generic models of human perception . |
| Approach: | They propose to use six machine-editing methods to model human perceptions of edited text . they use a dataset of human judgments of machine-edited text and scientific abstracts . |
| Outcome: | The proposed model is based on human judgments of machine-edited text and scientific abstracts . human judgment of edited text is predicted to be within 6% of human consensus labeling . |
Enhancing Variational Autoencoders with Mutual Information Neural Estimation for Text Generation (D19-1)
Copied to clipboard
| Challenge: | Existing approaches to train variational autoencoders (VAEs) have been proposed to alleviate the posterior collapse issue in NLP tasks. |
| Approach: | They propose to introduce a mutual information term between the input and its latent variable to regularize the objective of the VAE. |
| Outcome: | The proposed model performs better on three benchmark datasets and is comparable to state-of-the-art models. |
Spherical Latent Spaces for Stable Variational Autoencoders (D18-1)
Copied to clipboard
| Challenge: | Variational autoencoders use a multivariate Gaussian latent variable to capture latent structure in data. |
| Approach: | They propose a variational autoencoder which uses a latent distribution instead of Gaussian . they find that the variational posterior averts the KL collapse by a fixed hyperparameter . |
| Outcome: | The von Mises-Fisher distribution averts the KL collapse and gives better likelihoods than Gaussian models across a range of modeling conditions. |